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SOLUTION  OF  THREE  RELATED  COMBINATORIAL 
PROBLEMS  BY  NET\>JORK-FLOW  AND 
"BRANCH  AND  BOUND"  METHODS 


By 

Christopher  R.  Sprague 


ABSTRACT 

We  describe  a  highly  efficient  computational  procedure  for  the 

Assignment  Problem,  based  on  Ford-Fulkerson  Network-Flow  methods.   The 
Travelling  Salesman  Problem  can  be  viewed  as  an  Assignment  Problem  with 
side  constraints,  and  "Branch  and  Bound"  methods  due  to  Eastman  and 
Little  can  be  used  to  extend  the  Assignment  Problem  procedure  for  the 
Travelling  Salesman  Problem.   A  more  constrained  problem,  here  called 
the  "Travelling  Gourmet  Problem"  can  be  solved  by  straightforward 
extensions  to  the  Travelling  Salesman  procedure. 

We  present  several  alternative  algorithms  for  the  Travelling  Sales- 
man Problem  and  one  for  the  Travelling  Gourmet  Problem.   Computational 
results  are  Included. 
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Preface 

This  is  the  first  of  two  papers  about  methods  for  solving  a  par- 
ticular class  of  combinatorial  problems:   this  one  examines  the  Assignment 
Problem  and  Eastman's  sub tour-elimination  method  (modified  and  extended) 
for  the  Travelling-Salesman  Problem  and  a  close  relative,  the  Travelling 
Gourmet  Problem;  the  second  paper  discusses  the  Capacitated  Transportation 
Problem  and  its  relationship  to  the  special  (but  common)  case  of  symmetric 
Travelling  Salesman  and  Travelling  Gourmet  Problems.   Both  papers  stress 
the  synergy  between  Branch  and  Bound  techniques  for  evoking  subproblems, 
and  Network-Flow  methods  for  solving  them. 

This  first  paper,  however,  has  another  purpose  as  well.  The  algorithm 
presented  for  the  Assignment  Problem,  while  very  close  to  the  Standard 
Ford-Fulkerson  algorithm,  depends  heavily  on  a  set  of  techniques  which 
D.  C.  Carroll  has  called  "Semi -List  Processing."  The  term  is  not  well- 
defined,  but  refers  to  the  use  of  lists  of  pointers,  switches,  indicators, 
etc.  to  introduce  an  appropriate  amount  of  flexibility  into  data-handling, 
allowing  faster  computation.   In  this  specific  case  one  very  significant 
in5)rovement,  "reorganization,"  is  made  possible  only  by t he  fact  that  lists 
of  in^jortant  entities  are  maintained  at  each  stage  of  the  computation. 
Many  other  operations  also  depend  on  lists.   These  techniques  have  been 
conventional  wisdom  among  programmers  for  years  and  are  not  themselves 
particularly  exciting.   On  the  other  hand,  it  is  safe  to  say  that  the 
standard  mathematical  notation  used  to  express  algorithms  in  most  journals 


tends  to  obscure  the  best  choices  of  computational  methods. 


I  often  feel  that  the  more  elegantly  an  algorithm  is  stated,  the  more 
likely  it  is  that  the  statement,  straightforwardly  coded,  will  produce  a 
con5)utationally  inefficient  procedure.   This  is  not  to  say  that  the  ori- 
ginal designer  of  algorithms  should  present  them  as  machine-level  (or  even 
ALGOL)  code.   That  would  be  just  as  tragic  in  another  way.   It  does  mean 
that  many  insights  into  an  algorithm  can  be  obtained  from  attenuating  to 
produce  a  really  efficient  in^jlementation;  and  if  the  original  designer 
chooses  not  to  seek  high  efficiency  (and  sometimes  even  if  he  does)^  then 
there  is  a  distinct  need  for  others  to  try. 


*See,  for  example,  D.  N.  Ness,  "On  the  Computational  Efficiency  of 
Conventional  Notations,"  Sloan  School  Working  Paper    -70. 


I.   INTRODUCTION 

The  Problems 

This  paper  discusses  the  use  of  the  Network-Flow  methods  of  Ford 

1 

and  Fulkerson-^  in  the  solution  of  three  related  combinatorial  problems: 

2  3 

The  Assignment  Problem^   The  Travelling  Salesman  Problem,   and  the  Trav- 
elling Gourmet  Problem.   Each  of  these  problems  begins  with  an  m  x  m 
cost  matrix  C  =  >  c^^  (    and  asks  for  an  optimal  (least-cost)  permutation 
of  the  integers  1  .  .  .  m.  where  each  permutation  is  evaluated  as  follows: 
A  permutation  of  the  integers  1  .  .  .  m  consists  of  an  order- 
ing of  those  integers.   In  position  1  of  this  ordering  is  an 
integer  ii,    1  "^  Ji  £-  ^'    in  position  2  is  a  (different)  integer 

J2'  ^  ^  J2  -  ^''    ^''^^}    ^^   general,  position  i  contains  an  integer 

m 
Ji:.  1  i:  Ji  f  m.  The  cost  of  this  permutation  is  Z  =   <^  '^ii-' 

i=l   ^ 

Another  way  of  expressing  the  same  requirement:   Z  is  the  sum 
of  exactly  ra  elements  of  C  =  -I'  Cj^i    ,   where  exactly  one  element  is 
chosen  from  each  row  of  C  and  exactly  one  element  is  chosen  from 
each  colvimn  of  C. 

The  Assignment  Problem  allows  all  possible  permutations.   There  are 
thus  m.'  possible  solutions,  of  which  it  is  required  to  find  one  with  leaat 
cost  (there  may  be  ties). 

The  Travelling  Salesman  Problem  has  an  additional  constraint  often 
called  the  tour  constraint;   consider  each  integer  1  .  .  .  m  as  the  number 
of  a  city,  and  the  occurrence  of  j ^  in  position  i  of  the  permutation  as 
designating  a  trip  from  city  i  to  city  j^.   Then  the  pairs  of  integers 
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(i,  j  )  must  form  a  connected  tour  visiting  every  city  once  and  only  once. 
For  example,  in  a  5  x  5  problem,  the  permutation  p  =  (2, 5, A, 1,3)  repre- 
sents the  trips  (1, 2) (2,5) (3,4) (4  ,1) (5  ,3) .   Reorganizing  this  in  order  of 
visit  we  have  (1,2) (2 ,5) (5,3) (3,4) (A  ,1)  which  is  a  connected  tour  visit- 
ing each  city  once  and  only  once.   On  the  other  hand,  the  permutation 
p^  =  (2,5,4,3,1)  represents  the  trips  (1 ,2) (2,5) (3  ,4) (4  ,3) (5  ,1)  which 
becomes  two  disconnected  subtours:   (1 ,2) (2  ,5) (5  ,1)  and  (3,4) (4,3).   This 
latter  permutation  (p„)  i^  not  a  feasible  solution  to  the  Travelling 
Salesman  Problem  although  it  is^  a  feasible  solution  to  the  Assignment 
Problem.   There  are  thus  only  (m-1) I   possible  solutions  to  a  Travelling 
Salesman  Problem. 

The  Travelling  Gourmet  Problem  has  a  class  of  constraints  in  addition 
to  those  of  the  Travelling  Salesman  Problem  X'jhich  we  shall  call  cluster 
constraints.   For  simplicity  we  here  assume  only  one  such  constraint.   We 
designate  a  set  of  cities  K,  called  a  cluster,  and  an  associated  number 
S,  .   We  require  that  no  more  than  S   of  the  cities  in  K  be  visited  sequen- 
tially without  an  intervening  visit  to  some  city  not  in  K.   For  example, 
let  K  =  (2,3,5)  and  S,  =  2.   The  permutation  p,  =  (2,5,4,1,3)  which 
reorganizes  to  (1 ,2) (2,5) (5,3) (3,4) (4,1)  violates  the  cluster  constraint 
since  cities  (2,5,3)  are  all  visited  in  order  and  our  S  was  2.   The  permu- 
tation p   =  (2,4,5,3,1)  reorganizes  to  (1,2) (2,4) (4  ,3) (3  ,5) (5  ,1)  and  only 
2  of  the  cities  (2,3,5)  are  visited  in  order,  thus  satisfying  the  cluster 
constraint.   While  we  have  assumed  only  one  such  constraint,  generalization 
to  two  or  more  is  straightforward.   There  are  less  than  (m-1) I  possible 
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solutions  to  a  Travelling  Gourmet  Problem  and,  indeed,  it  is  very  easy 

to  define  a  set  of  cluster  constraints  which  reduce  the  number  of  possible 

solutions  to  zero.   For  example,  try  a  5  x  5  problem  with  K  =  (2,3,4,5), 
S   =  2 

Applications 

Applications  for  all  three  problems  range  from  the  fanciful  to  the 
practical.   While  the  Assignment  Problem  has  been  suggested  for  matching 
brides  to  grooms,   it  is  used  to  match  jobs  to  machines.    The  Travelling 
Salesman  Problem  may  have  actually  been  used  for  scheduling  salesmen, 
but  more  often  for  sequencing  jobs  through  machines.   The  Travelling 

Gourmet  Problem  was  conceived  to  allow  a  gourmet  to  visit  most  economl- 

g 
cally  all  of  the  three-star  restaurants   in  France  without  eating  the 

same  regional  specialties  more  than  twice  in  succession.   It  has  obvious 

application  to  the  scheduling  of  sports  teams,  dispatching  of  trucks, 

performance  of  tasks  in  contaminated  environments,  etc. 
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II.   SOLUTION  OF  THE  ASSIGNMENT  PROBLEM  BY  FORD-FULKERSON  NETWORK-FLOW 
METHODS 

Theory 

The  following  is  a  discussion  of  a  version  of  Ford  and  Fulkerson's 

Primal-Dual  Method  for  the  Transportation  Problem^  •'■^  especially  adapted 

for  the  Assignment  Problem  and  particularly  well-arranged  for  automatic 

computer  solution.   We  begin  by  noting  that  the  Assignment  Problem  may 

be  restated  as  follows: 

Find  an  m  x  m  matrix  X  "^  /•  '<^ii')^ 

m    m 
x^.   =  0^1  all  i^j   such  that  Z  =  ^   •'-   c^^^x^^ 


1=1  j=l 


is  minimized^  subject  to: 


'>   x^-   =  i^  all  i 
i=l 


m 
<  X..  =  1,  all  j, 
1=1 


By  duality^  this  is  equivalent  (except  for  the  integer  constraint  on 
the  X. . 's)  to: 

Find  vectors   U  =  )  uA  ,        V  =  )  Vi  '  such  that 

m       m 

W  =  '^^  u^  +  J2:   v^  is  maximized, 
i=l      j=l 

subject  to:   Uj^  +  Vi  ^  '^il^  ^^^   ^!    J- 

By  complementary  slackness,  an  optimal  solution  to  both  the  primal 
and  dual  problems  will  occur  only  when  Xj^4   =  0  everywhere  that  u^^  +  Vi  <!  c^^ 

Our  procedure  is  then  to  form  a  feasible  dual  solution  (an  easy  thing  to  do 
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—  see  below)  and  see  if  a  feasible  solution  to  the  primal  can  be  found 
allowing  only  those  x^j's  where  Uj|^  +  v^   =  c^^^l    to  differ  from  zero.   This 
so-called  "restricted  primal"  takes  the  siii^jle  form  of  maximizing  the  flow 
in  a  network  defined  by  those  Cj^j's  equal  to  u^  +  v j  .   If  the  restricted 
primal  is  feasible,  we  have  an  optimal  solution.   If  not,  the  results  of 
the  attempt  at  primal  feasibility  are  used  to  derive  a  new  (higher  cost) 
dual  and  the  process  is  repeated.   One  pleasant  consequence  of  this  method 

is  that  the  resulting  x^j's  inevitably  are  0  or  1;  another  is  that  the 

m       m 

value  of  W  =  ^^  u^^  +  ^    v^  rises  monotonically  throughout  solution,  of 

i=l      i=l 
which  more  later. 

In  reaching  a  solution,  we  shall  make  use  of  eight  m-vectors,  as  fol- 
lows: T  =  \^±i   and  F  =  ffiV  take  the  place  of  x  =  S  x^j\ .  Since  no 
more  than  one  x^^  will  differ  from  zero  in  each  row  and  in  each  column, 
the  entire  matrix  X  can  be  represented  thus: 

If  no  Xj^4  in  row  i  differs  from  zero,  then  tj^  =  0.  But 

if  some  x^^j  in  row  i  equals  1,  then  t^  =  j .  Similarly,  if  no 

x^^  in  column  j  differs  from  zero,  then  f-  =0.   But  if  some 
Xj^i  in  column  j  equals  1,  then  fj  =  i. 

U  =  t  "i  (  iind  V  =  i  V::  I  are  the  dual  solution  spoken  of  above;  T,F,U,V 
and  W  represent  the  entire  primal  and  dual  solutions. 

In  order  to  understand  the  use  of  the  other  four  m-vectors,  a  conmon 
vocabulary  must  be  established.   A  row  i  will  be  said  to  be  labeled  from 
the  source  iff  ti  =  0,  i.e.,  no  x-j^j  differs  from  zero.   A  colvmm  j  will 
be  said  to  be  labelable  from  row  _i  iff  row  i  is  labeled  and  c^^j   =  u^  +  v^ . 
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While  a  column  may  be  labelable  from  many  different  rows^  it  can  at  any 
time  be  labeled  from  no  more  than  one.   A  row  i  is  said  to  be  labeled  from 
column  i  iff  column  j  is  labeled  and  x^^   =  1,  i.e.^  £z      =     i. 

R  =  )  r j^  I  contains  the  row  labels:   if  row  i  is  unlabeled,  r^r   =  0; 
if  row  i  is  labeled  from  the  source,  r^  =  -1;  and  if  row  i  is  labeled  from 
column  j,  r-j^  =  j.  P  =  !  Pj  (  contains  the  column  labels:   if  column 
j  is  unlabeled,  p-   =  0;  and  if  column  j  is  labeled  from  row  i,  p^   =  i. 
At  any  point  in  the  computation,  some  rows  are  labeled  and  others  are  not. 
The  first  n^   elements  of  S  =  ^  ^q  (  ^^^  *^^^  indices  of  labeled  rows.   Simi- 
larly, some  columns  are  labeled  and  the  others  are  not.  The  first  n^  ele- 
ments of  D  =  [^  djj  ^  are  the  indices  of  labeled  columns,  while  the  last 

m  -  n^  elements  of  D  are  the  indices  of  unlabeled  columns. 

m       m 
It  was  noted  previously  that  the  value  of  W  =  ^     "i  +  ^T  v^  rises 

i=l      i=l  -^ 
monotonically  throughout  the  computation.   We  shall  find  it  useful  later 

to  be  able  to  suspend  the  conqjutation  if  W  becomes  equal  to  or  greater  than 
some  arbitrary  value  B.   The  algorithm  described  below  includes  this  capa- 
bility. 

ForminR  an  Initial  Solution 

We  begin  by  finding  an  initial  solution  which  consists  of  a  con5>lete 

mm 
feasible  dual  (W  =  -^     "i  "^  £1  ^i  ^"'^  '^ii  '"  "i  "*"  '^i^  ^'^'^   ^'^^   ^^'^  ^ 

i=l      j=l 
partial  primal  (at  least  one  x^^  =1): 

min 


1.  For  every  i,  set  u^  =  j   (c^j) 

set  t^  =  0 

„   _,         .  min  ,  » 

2.  For  every  j,  set  v^  =  .   (cj^j-ui) 


set  f j   =  0 
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m       m 
3.   Set  W  =  ^   Ui  +  £^  Vj 
i=l      j=l 


4.  For  every  i,  j  where  c^j   =  Ujj^  +  Vj^  and  t^  =  0  and  f^  =  0^ 

set  t^  =  j  and  set  f::   =  i 

5.  For  every  h,  set  d,   =  h. 

Set  Initial  Labels 

We  now  set  all  columns  unlabeled^  and  set  all  rows  without  assignments 
(ti  =  0)  labeled  from  the  source.   Rows  with  assignments  are  set  un- 
labeled,  S  will  contain  a  list  of  labeled  rows,  and  ng  will  be  its  length. 
D  contains  a  list  of  columns  and,  since  all  are  unlabeled,  n-,  will  be 
zero. 

6.  Set  n^  =  0 
Set  ng  =  0 


7.   For  every  i,  set  r^  =  0 


8.   For  every  i  where  t^  =  0, 


Set  r^  =  -1 


Set  ng  =  ng  +  1 


Set  Sjj  =  i 
s 


9.   For  every  j,  set  p.   =  0 


10.   Set  q   =  0. 
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Feasibility  Check 

We  now  check  to  see  whether  all  rows  have  assignments.   If  so,  we 
are  finished,  and  we  return  with  the  x. .'s  in^jlicitly  contained  in  T  and 
F.   W  contains  the  objective  function  value, 

11.  If  n   =  0,  return 

s      ' 

Bogey  Check 

We  now  check  to  see  whether  W  <   B.   If  not,  we  return  with  an  in- 
con5)lete  solution. 

12.  If  W  ^  B,  return. 

Primal  Continuation  Check 

We  now  increment  q  (the  index  of  s,    set  in  step  10  or  37)  and  test 
whether  there  remains  a  labeled  row  to  scan.   If  so,  we  continue,  but  if 
not,  a  revision  of  the  dual  is  required. 

13.  Set  q   =  q  +  1 

If  q   >  ng,  go  to  step  30. 

Attempt  to  Label  Columns 

Here  we  scan  a  labeled  row  i  =  s  to  see  whether  an  unlabeled  col- 
umn j  exists  for  which  c^^   =  Uj^  +  v^ .  If  so,  this  column  can  be  labeled. 
If  not,  we  return  to  step  13  for  another  labeled  row. 

14.  Set  i  =  s 

q 

Set  h  =  n^ 

15.  Set  h  =  h  +  1 
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16,  If  h  >  m,  go  to  step  13 
Set  j   =  dj^ 

If  c^j   /  Uj^  +  Vj^  go  to  step  15. 

Label  a  Coltmin 

Here  we  label  the  column  found  in  step  16.   If  this  column  already 
has  an  assignment  (fj   ^  0)  we  can  label  a  new  row  (fj)  and  continue 
searching  row  i  for  labelable  columns.   If  not^  we  can  make  a  new  assign- 
ment.  Step  17  labels  column  j,  reorders  d-^,    and  increments  n^   to  preserve 
the  convention  that  the  first  n^   elements  of  D  call  out  labeled  columns, 
while  the  last  m  -  xy  elements  refer  to  unlabeled  columns.   Note  that  we 
only  search  the  latter  part  of  D  in  this  procedure. 

17.  Set  n J  =  n^  +  1 


•d 


Set  d^     =  d„ 

Set  d„  =  j 
"d 

Set  pj   =  i 
18.   If  f j   =  0,  go  to  step  20. 

Set  a  New  Row  Label 

Here  we  label  a  new  row,  specifically  the  row  in  which  the  just-labeled 
column  j  has  an  assignment.   This  is  perhaps  a  good  point  at  which  to  men- 
tion the  physical  significance  of  labels.   A  row  labeled  from  the  source 

m 
is  one  which  has  no  assignment  in  it  (i.e.,  tj^  =  0,  or  <^  x^^  =  0) 

i=l 

A  column  j  labeled  from  row  i  indicates  one  of  two  things:   if  column  j 

has  no  assignment,  one  could  be  placed  in  row  i;  if  column  j  has  an  assign- 
ment y  "  f  ■_,  it  could  be  moved  from  row  y  to  row  i.   A  row  y  labeled 
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from  column  j  already  has  an  assignment  in  column  j,  but  another  could  be 
made  by  moving  the  assignment  in  column  j  to  row  p^. 

For  example,  suppose  row  1  is  labeled  from  the  source  and  column  5  is 
labeled  from  row  1.   Suppose  also  that  f^  =  3,  which  means  that  row  3  is 
labeled  from  column  5.   Now  suppose  that  column  2  is  labeled  from  row  3, 
but  f2  =  0,  as  in  the  following  table: 


Row  or  Column 

ti 

^i 

^j 

Pj 

1 

0 

-1 

NA 

NA 

2 

m. 

NA. 

0 

3 

3 

5 

5 

m 

NA 

4 

m 

NA 

NA 

NA 

5 

m. 

NA. 

3 

1 

This  means  that  an  assignment  could  be  placed  in  row  3,    column  2. 
There  would  then  be  two  assignments  in  row  3,  but  row  3  is  labeled  from 
column  5,  and  column  5  is  labeled  from  row  1^  so  we  move  the  assignment 
in  row  3^  column  5  to  row  I,    column  5.   Row  1  was  labeled  from  the  source, 
so  it  had  room  for  an  assignment. 

Step  19  puts  a  label  on  the  row  in  which  a  labeled  column  has  an 
assignment. 


19.   Set  y   =  f . 
Set  ry  =  j 
Set  n^   =   n3  +  1 


Set  Sj,^  =  y 
So  to  step  15. 
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Make  a  New  Assignment 

Here  we  follow  the  procedure  outlined  above  to  make  a  new  assignment, 
following  the  chain  of  labels  back  to  the  source.   As  we  go  back  to  the 
source,  we  set  each  row  label  in  the  chain  to  zero,  to  indicate  that  it 
is  no  longer  valid. 


20.   Set  i  =  pj 


Set  ti  =  j 

Set  £j  =  i 

Set  j  =  r^ 

Set  v^  =  0 

21.  If  j  ^     -I,    go  to  step  20. 

Unwind  Labels 

Here  we  contract  the  lists  of  labels  to  account  for  those  no  longer 
valid  because  of  a  new  assignment.   The  straightforward  way  of  coping  with 
this  problem  is  to  effectively  discard  all  labels  by  returning  to  step  6, 
but  considerable  computation  may  have  gone  into  producing  those  labels 
some  of  which  would  simply  reappear,  so  some  effort  is  called  for.   Basic- 
ally, we  have  two  loops.   The  first,  steps  22-26,  removes  the  labels  from 
columns  which  have  no  intact  chain  back  to  the  source.   The  second,  steps 
27-29A,  removes  the  labels  from  rows  which  are  labeled  from  unlabeled  col- 
umns, and  it  also  removes  unlabeled  rows  from  S.   At  the  end  of  both,  we 
return  to  step  10. 

22.  Set  h  =  0 

23.  Set  h  =  h  +  1 
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24.  If  h  >    n^,    go  to  step   27 
Set   J  =     djj 

25.  Set   i  =     Pj 

If  i  =  0,  go  to  setp  26 

Set  j  =  ri 

If  J  =  -1.  go  to  step  23 

If  J  ^  0,  go  to  step  25 

26.  Set  j  =  di^ 


Set  dh  -  d„ 


Set  d„^  =  j 


Set  p. 


0 


Set  n^j  =  i\j  -  1 
Go  to  step  24 

27.  Set  a  =  0 
Set  q  =  0 

28.  Set  q   =   q  +  1 

If  q   7  n  ,  go  to  step  29A 

Set  i  =  s^ 

If  r^  =  0,  go  to  step  28 

If  Tj^  =  -1  go  to  step  29 

Set  j   =  r^ 

If  Pj  i"     0,  go  to  step  29 

Set  r^  =  0 
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29.   Set  a  =  a  +  1 
Set  s^  =  i 
Go  to  step  28 

29A.  Set  iig  =  a 
Go  to  step  10. 

Revise  Dual 

Now  we  revise  the  dual  solution,  based  on  the  primal  solution.   We 
came  here  from  step  13  and  at  this  point  no  more  columns  can  be  labeled 
from  labeled  rows.   In  order  to  make  at  least  one  more  Cj^i      =  u^^  +  v^  in 
some  labeled  row,  we  must  add  some  number  to  the  u^^'s  of  all  labeled  rows. 
But.  to  preserve  the  current  assignments  and  labels,  we  must  subtract  the 
same  number  from  the  v, 's  of  all  labeled  columns.   Our  first  task  is  to 

■J 

find  that  number,  which  is: 

min  (c£j  -  u^  -  Vj) 
(i  labeled,  j  unlabeled) 

We  will  also  make  a  list  of  all  rows  in  which  this  minimum  value  occurs. 
This  will  save  us  considerable  computation  when  we  restart  the  primal.   In 
order  to  do  this,  we  need  a  working  m-vector  E  = > e_ ^  and  a  length  n^. 

30.  Set  q  =  0 
Set  a  =  "^^ 

31.  Set  q   =   q  +  1 

If   q   >■  r^ ,  go  to  step  34 
Set  h  =  nd 

^"^  ^   =   ^q  -  13  - 


32.   Set  h  =  h  +  1 

If  h  >  m^  go  to  step  31 

Set  j  =  d^ 

If  a  <  c^j  -  u^  -  v^,  go  to  step  32 


If  a  =  c, 


ij 


"i  "  "^jj  8°  ^°  step  33 


Set  iig  =  0 


Set  a  =  c^j  -  u^  -  Vi 


33.   Set  Rg  =  rig  +  1 


Set  e^^  =  q 
Go  to  step  32. 


Revise  W 

SOj  a  is  the  number  to  be  added  to  all  u^'s  (i  labeled)  and  subtracted 
from  all  v^'s  (j  labeled). 

34.  For  every  q^,  1  ^     q  •-  ng 

Set  i   =  Sq 
Set  u^  =  u^  +  a 

35.  For  every  h.  1   f  h  '^  nj 

Set  j    =  dh 
Set  vj   =  Vi  -  a 

36.  Set  W  =  W  +  (ng  -  n^)    "  a 

Reorganize  S 

We  have  revised  the  dual  solution,  and  can  begin  to  label  again. 
However,  only  those  rows  whose  q-indices  are  called  out  in  E  contained 
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'^ii  "  u^  -  Vj   =  a,    and  only  those  rows  will  now  contain  c^i    -  u-j^  -  v^   =  0. 
Therefore  we  reorganize  S,  placing  these  eligible  rows  at  the  end,  setting 
q  so  that  a  return  to  step  12  will  restart  the  primal  in  such  a  way  that 
only  these  eligible  rows  will  be  scanned. 

37,  Set  q  =  ng 

For  all  g,  1  -^   g  ^   n^ 
Set  a  =  Sq 

Set  s„  =  Sp 

q      Kg 

Set  s„  =  a 
Set  q   =  q  -  1 

38.  Go  to  step  12. 

This  coii5)letes  our  description  of  a  network-flow  procedure  for  the 
assignment  problem.   Readers  familiar  with  the  standard  versions  of  similar 

algorithms  (such  as  for  the  transportation  problem)  will  note  some  differ- 
ences . 

First,  the  major  loops  of  the  procedure  are  concerned  with  setting 
labels.   Consequently,  much  care  is  taken  to  see  that  no  label  is  ever 
discarded  unless  it  must  be  discarded. 

Second,  search  is  reduced  to  a  minimum  by  the  use  of  what  Carroll  has 
termed  "semi-list"  processing.    For  example,  X  is  compressed  into  T  and 
F,   The  procedure  maintains  lists  of  labeled  rows  and  columns  so  that  the 
labels  themselves  are  rarely  tested.   The  keeping  of  lists  also  makes  possi- 
ble the  reorganization  performed  in  steps  37-38,  which  again  results  in  a 

lessening  of  search  time. 
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Third,  as  long  as  the  current  state  of  U,  V^  T^  F  and  W  is  dual- 
feasible,  the  procedure  can  be  safely  (and  profitably)  entered  at  step  6. 

We  now  describe  a  procedure  for  changing  any  given  c^=  while  maintain- 
ing dual  feasibility. 

Changing  cij 

We  enter  with  i^  j  and  a,    the  new  value  of  c^j . 

41.  If  a  =  Cij,  return 

If  a  <1  c .  .  go  to  step  44 

42.  Set  c. .   =  a 

If  t^  ^     j,  return 
Set  y  =  j 


43.   Set  t^  =  0 


Set  f   =  0 
Return 


44.   Set  c^j   =  a 

If  c^.  -  Uji^  -  Vj  ^-    0  return 


45.  Set  W  =  W  +  c^j  -  u^  -  Vj 
Set  Ui  =  Cij  -  Vj 

If  t^  =0,  return 

46.  Set  y  =   tj^ 
Go  to  step  43, 
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Computatioml  Experience 

The  following  times,  in  seconds,  were  observed  for  20  runs  of  the 
Assignment  Problem  procedure  described  above.   Runs  were  made  on  an  IBM 
1130  located  at  the  Sloan  School  of  Management,  M. I.T.   Source  language  was 
FORTRAN. 
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Table  I 


PROBLEM     TYPE 


Method 


SIZE 

0 

1 

2 

3 

20 

4 

4 

4 

4 

20 

6 

5 

6 

5 

40 

27 

19 

28 

20 

40 

31 

21 

31 

22 

60 

89 

66 

94 

67 

60 

85 

59 

86 

59 

80 

175 

119 

174 

120 

80 

206 

149 

205 

149 

100 

255 

185 

251 

180 

100 

252 

179 

250 

167 

20 

1 

1 

1 

1 

20 

3 

2 

3 

2 

40 

11 

10 

12 

10 

40 

14 

10 

14 

10 

60 

86 

59 

85 

59 

60 

22 

19 

20 

19 

80 

115 

84 

112 

82 

80 

65 

50 

64 

49 

100 

195 

142 

192 

140 

100 

123 

92 

119 

91 

1  RECT 

2  " 
3 

4 

5  '• 

6  " 

7  " 

8  " 
9 

10  " 

11  EUCL 
12 

13 

14 

15        " 

16 

17 

18 

19 

20 


18 


Key: 

Type  =  RECT  —  Costs  were  random  3-digit  rectangularly  distributed 
integers  0-999.   All  diagonals  were  set  to  ^  . 

Type  =  EUCL  —  Costs  were  linear  distances  rounded  to  integers^ 
between  points  located  on  a  cartesian  plane.   Points  were  located  randomly 
be  choosing  integer  X  and  Y  coordinates  rectangularly  distributed  in  the 
range  0-999.   All  diagonals  were  set  to  <>»  , 

Method  =  0  —  As  described,  but  no  unwinding  (steps  22-29A)  and 
no  reorganization  (steps  37-38). 

Method  =  1  —  As  described,  but  no  unwinding 

Method  =  2  --  As  described,  but  no  reorganization 

Method  =  3  —  As  described. 

We  can  see  that  reorganization  appears  to  reduce  the  required  time  by 
257o  or  more.  Unwinding,  on  the  other  hand,  seems  to  be  of  value  primarily 
for  larger  problems,  and  is  of  marginal  help  even  then.   But  we  expect 

unwinding  to  help  more  for  larger  problems,  because  rebuilding  labels  must 

2 
be  an  m  process,  while  unwinding  itself,  as  can  be  seen  from  the  descrip- 

9 

tion.  should  go  as  less  than  m'^. 

To  test  the  ability  of  the  Assignment  Problem  procedure  to  re-solve 
after  changes  in  C,  the  following  was  tried: 

After  a  problem  was  solved,  one  row  was  chosen  at  random  and  the  c^:: 
corresponding  to  the  assignment  in  the  chosen  row  was  changed  to  ^''^   . 
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The  problem  was  then  solved  again.   Then  another  different  row  was  chosen, 

another  '^  put  into  it,  and  so  on  until  m  Cj^i's  —  one  in  each  row  — 

had  been  set  to  O'  and  the  problem  had  been  solved  a  total  of  ra  +  1 
times. 

The  process  was  then  reversed  and  each  Cj^j  was  restored  to  the  ori- 
ginal value,  again  in  random  order,  and  again  with  a  new  solution  after 
each  change,  for  a  total  of  2m  +  1  solutions.   Table  II  summarizes  the 
results.   The  problems  are  a  subset  of  those  in  Table  I.   T-j^  is  the  time 
for  the  first  solution  and  corresponds  exactly  to  solution  time  by  method 
3  in  Table  I.   T2  is  the  mean  time  for  each  re-solution  after  an  upward 
revision  in  a  c^^.  T-j  is  the  mean  time  for  each  re-solution  after  a 
downward  revision. 
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Table 

II 

PROBLEM 

TYPE 

SIZE 

Tl 
4 

T2 
.75 

T3 

1 

RECT 

20 

.40 

2 

11 

20 

5 

.65 

.40 

3 

II 

40 

20 

2.33 

1.85 

4 

II 

40 

22 

2.05 

1.15 

5 

11 

60 

67 

5.17 

5.00 

6 

II 

60 

59 

4.73 

3.42 

11 

EUCL 

20 

1 

.55 

.35 

12 

II 

20 

2 

.65 

.45 

13 

II 

40 

10 

2.08 

.88 

14 

II 

40 

10 

2.00 

1.25 

15 

II 

60 

59 

3.58 

2.12 

16 

II 

60 

19 

3.00 

1.58 

There  is  clearly  a  substantial  advantage  to  restarting  the  Assign- 
ment Problem  procedure  with  a  partial  solution.   This  will  prove  to  be  of 
significant  value  in  our  attack  on  the  Travelling  Salesman  Problem  in  the 
next  section. 
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III.   SOLUTION  OF  THE  TRAVELLING  SALESMAN  PROBLEM 
BY  VARIANTS  OF  EASTMAN'S  SUBTOUR-ELIMINATION  METHOD 


Though  the  Travelling  Salesman  Problem  has  fewer  solutions  than  does 
the  equivalent  Assignment  Problem,  it  has  proven  much  harder  to  solve.   Of 
exact  methods  for  its  solution,  the  "Branch  and  Bound"^-'^  methods  of  Little 
et.  al.,   Eastman^-*  and  Shapiro-"-^  seem  most  useful  for  problems  of  non- 
trivial  size.   We  begin  by  describing  the  elements  of  Branch  and  Bound 
methods  for  the  Travelling  Salesman  Problem. 

Branch  and  Bound 

We  begin  with  the  original  problem  which  we  mark  "open."  We  also 
establish  a  best  solution  value  or  "bogey"  equal  to  '>^  . 

1.  Choice:   Choose  some  open  problem  (initially,  there  is  only  one). 
Mark  it  "closed." 

2.  Attempt  to  Solve:   Take  the  problem  chosen  in  step  1  and  attempt 
to  solve  it  "by  inspection."   If  it  is  possible  to  do  so,  go  to 
step  5,   If  not,  go  to  step  3. 

3.  Branching:   By  some  method,  divide  the  problem  chosen  (now  called 
the  parent  problem)  into  two  or  more  new  problems,  each  of  which 
is  "easier"  than  the  parent  problem,  but  which,  among  them  all, 
contain  every  solution  feasible  in  the  parent.   Mark  all  these 
new  problems  "open." 

4.  Bounding:   Examine  each  of  the  new  open  problems  produced  in 
step  3  and  assign  to  each  a  lower  bound  on  the  value  of  the  best 
solution  in  cash.   If  any  such  lower  bound  equals  or  exceeds 
"bogey,"  discard  the  new  problem  immediately.   When  finished,  go 
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to  step  7. 

5.  Bogey  test:   If  the  value  of  the  solution  found  in  2  is  strictly 
less  than  "bogey",  go  to  Step  6.   Otherwise  go  to  Step  7. 

6.  Record  New  Bogey:   Set  "bogey"  to  the  value  of  the  solution 
found  in  2.   Record  the  solution  itself  as  "tour."  Now  look  at 
all  problems  remaining  and  discard  any  problem  whose  lower  bound 
as  found  in  step  4  is  greater  than  or  equal  to  "bogey." 

7.  Test  for  con^jletion:   If  there  remain  any  open  problems^  return 
to  step  1.   Otherwise  terminate.   "Tour"  is  the  optimum  solution, 
and  'Tjogey"  is  its  value. 

Obviously,  Branch  and  Bound  methods  can  be  used  for  problems  other 
than  the  Travelling  Salesman  Problem.   For  the  present,  however,  we  confine 
our  attention  to  the  Travelling  Salesman  Problem,  and  we  now  proceed  to 
specialize  the  description  above  to  Eastman's  Method,  which  depends  on 
the  observation  that  the  value  of  an  Assignment  Problem  solution  under  a 
matrix  C  is  a  lower  bound  on  the  value  of  a  Travelling  Salesman  Problem 
solution  under  the  same  matrix. 

Eastman's  Method 

Begin  with  the  original  m  x  m  cost  matrix,  with  all  diagonal  elements 
set  to  ''^    .   Mark  it  "open."  Set  its  lower  bound  to  0.   Set  'Taogey"  to 


,^-Ci 


El.   Choice:   Choose  the  open  problem  with  lowest  lower  bound. 
Mark  it  "closed." 

E2.   Attempt  to  solve:   Submit  the  chosen  problem  to  the  Assignment 
Problem  Procedure  described  in  Section  II  with  B  =   "bogey." 
Set  the  lower  bound  of  this  problem  to  W.   If  W     B,  there 
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can  be  no  Travelling  Salesman  Problem  solution  value  <'  "bogey" 
so  we  go  to  step  E7.   If  W  -^  B,  we  examine  the  Assignment 
Problem  solution.   If  the  x^^j's  form  a  tour,  we  go  to  step  E6. 

E3.   Branching:   If  the  x^^'s  do  not  form  a  tour,  they  form  two  or 
more  subtours  (connected  sequences  of  cities  shorter  than  m) . 
Choose  the  shortest  subtour.   It  contains,  say,   q(2     q     m/2) 
non-zero  x . . 's.   Create  q  new  open  problems  each  of  which  differs 
from  the  parent  problem  by  having  a  (different)  c^^  correspond- 
ing to  a  non-zero  x^s   on  the  subtour  set  to  "*■'  .      If  c^^   =  -^  , 
x^:.    is  said  to  be  "barred."  Note  that  in  none  of  these  new 
problems  can  the  same  subtour  recur  (one  of  the  c^^'s  is  now  ^^'  ) 
but  any  tour  (which  must  contain  no  more  than  q  -  1  of  these 
Xj^s's)  can  occur  in  at  least  one  of  the  new  problems. 

E4.   Bounding:   Set  the  lower  bound  of  each  of  the  new  problems  to 
the  lower  bound  of  the  parent  problem.   Go  to  step  E7. 

E5.   Bogey  test:   This  step  has  been  absorbed  into  step  2. 

E6.   ilecord  new  bogey:   Set  "bogey"  =  W.   Set  "tour"  =  T.   Discard 
all  problems  whose  lower  bound  equals  or  exceeds  bogey. 

E7.   Test  for  con^iletion:   If  there  are  still  one  or  more  open 
problems,  go  to  step  El.   Otherwise,  terminate. 

This  procedure  requires  relatively  little  space  in  a  computer.   One 
obviously  needs  to  store  the  matrix  C.   But  for  each  problem  to  be  stored, 
one  needs  only  space  to  store  the  lower  bound,  the  position  of  the  parent 
problems,  the  position  of  the  x^j^-  to  be  barred,  and  the  original  value 
of  the  c-.  replaced  by  <^'^   . 
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Changing  C  from  problem  x,  say^  to  problem  y,  requires  finding  their 
lowest  level  common  ancestor^  call  it  z.   We  can  then  move  up  from  problem 
X  to  z,    parent  by  parent,  restoring  c^j's  on  the  way.   Then  we  move  up 
from  y  to  z,  parent  by  parent,  barring  x^^'s. 

Use  of  the  procedure  for  changing  c   's  described  in  Section  II,  we 

ij 

maintain  dual  feasibility,  so  that  at  no  time  after  the  first  is  a  com- 
plete solution  of  the  Assignment  Problem  required. 

Shapiro  found  a  method  essentially  identical  to  Eastman's  very  effect- 
ive for  Travelling  Salesman  Problems  of  the  type  here  called  RECT.    Never- 
theless, there  are  two  relatively  sin5)le  extensions  to  Eastman's  Method 
which  are  worth  considering.   We  proceed  to  describe  them. 

Example  of  a  Solution  by  Eastman's  Method 


x= 


J= 

'•v_ 

1 

2 

3 

4 

5 

1 

^-"-' 

8 

4 

0 

5 

2 

8 

-■  -^ 

2 

6 

2 

3 

4 

2 

r  -o 

7 

3 

4 

0 

6 

7 

-^ 

1 

5 

5 

2 

3 

1 

--. 

One  initial  tree  of  solutions  has  only  one  problem  (node)  called  "all 
tours,"  with  lower  bound  0  "Bogey"  is 


All  tours  „ _  (j-o 


0 

,  open 


Bogey 
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Step  El  —  We  choose  the  only  open  problem  ("ALL  TOURS")  and  mark  it 
closed. 

Step  E2  --  We  use  the  Assignment  Problem  procedure  and  obtain: 
W  =  7,  T  =   (4,5.2,1,3),     F  =   (4.3,5,1,2) 
U  =   (2,2,3,0,3),    V  =  (0,-1,0,-2,0) 

This  corresponds  to  two  subtours,  (1,4)(4,1)  and  (2,5) (5,3) (3,2) .   We 
go  to: 

Step  E3  —  We  create  2  new  open  problems,  in  the  first,  called  A,  we 
bar  X]^/^;  in  the  second,  called  B,  we  bar  xa]^. 

Step  E4  —  Set  the  lower  bounds  on  both  A  and  B  to  W  which  is  7.   Our 
tree  is  now  , 


/ 

f  All  Tours  I     ,    , 
i j    closed 

_   /         \_ 

4,iy  \i,4 


Bogey 


=  (?o 


/ 

B 

V 

\ 

\ 

7 

'  open 

/ 


— - —  j  open 
/ 

Step  E7  —  There  are  two  open  problems,  A  and  B. 

Step  El  --  We  choose  problem  A  and  mark  it  closed.   Using  the  procedure 

for  changing  one  c  .  .,  we  set  c, ,   =   '^^  .   This  yields 

W  =  9,   T  =  (3,5,2,1,4),   F  =   (4,3,1,5,2) 

U  =   (4,2,2,0,1),    V  =   (0,0,0,0,0) 
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This  is  a  tour  (1 ,3)(3,  2)  (2,5)  (;5,4)  (4, 1) 
We  go  to: 

Step  E6  —  Set  bogey  to  9,    set  tour  to  (3,5,2,1,4). 

Step  E7  --  There  is  one  open  problem,  B. 

Step  El  —  Choose  problem  B  and  mark  it  closed.  This  requires  two  steps. 
First  set  the  value  of  c-^^  to  0  (its  original  value).  Then  set  the  value 
of  c^j^  to  ^  .   This  yields: 

W  =  5,   T  =   (0,5,2,0,4),   F  =   (0,3,0,5,2) 

U  =  (0,2,2,0,1),   V  =   (0,0,0,0,0) 

Step  E2  —  Start  Assignment  Problem  procedure  at  step  6.   It  returns  early 
with  W  =  bogey  =  9.  - 

Step  E7  —  There  are  no  more  open  problems.  Tour  is  (3,5,2,1,4).   The 
cost  is  9. 

Eliminating  Redundant  Problems 

Unfortunately,  the  branching  procedure  in  Step  E3  produces  a  set  of 
new  problems  which  are  not  mutually  exclusive.   In  particular,  any  tour 
containing  no  more  than  q  -  2  of  the  non-zero  x.  .'s  making  up  the 
subtour,  is  legal  in  all  the  new  problems.   We  will  describe  Little's 
solution  to  this  problem  later;  here  we  describe  a  method  which  fits 
within  Eastman's  framework. 

Observe  that  if  there  be  two  open  problems  y  and  z  such  that  the 
barred  x-.'s  in  y  are  a  subset  of  the  barred  x^:j's  in  z,  then  any  tour 
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possible  in  z  is  also  possible  in  y;  there  is  no  need  for  storing  or  con- 
sidering z.   In  practice,  it  is  difficult  to  find  all  such  subset  rela- 
tionships, but  the  following  procedure  finds  a  great  many  of  them: 

As  each  new  problem  is  created^  we  ask  if  any  open  "uncle"  of  the 
problem  has  the  same  barred  x^^^.   If  so,  the  new  problem  differs  from  its 
uncle  only  by  having  one  more  barred  y^^i,    namely,  that  of  its  immediate 
parent.   It  is  therefore  redundant.   If  not,  we  try  great-uncles,  great- 
great-uncles,  and  so  on.   The  diagram  below  shows  the  relationships  in- 
volved.  Circles  represent  problems,  while  the  numbers  in  them  represent 
barred  x^^::  's. 


Redundant  problem 


Barring  Multiple  ^-cs's 


Eastman  makes  the  following  elegant  argument: 

Consider  a  subtour  of  length  q  where  q  >  2,   In  order 
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for  the  tour  constraint  to  be  met,  at  least  one  city  on  that 
subtour  must  eventually  be  assigned  to  some  city  not  on  that 
subtour.   Therefore,  when  creating  a  new  problem  corresponding 
to  an  Xij  on  the  subtour,  it  is  permissible  to  bar  also  other 
xij's  corresponding  to  assignments  of  city  i  to  other  cities  on 
the  subtour.   For  example,  consider  the  subtour  (1,2)(2,3) (3, 5) (5, 1) . 
The  first  new  problem  would  normally  bar  x-[^2-  ^^   ^^   also  per- 
missible to  bar  x-]^3  and  x-j^^  in  this  new  problem. 

Barring  a  variable  number  of  x..'s  poses  some  problem  in  computation. 
But  barring  two  x.  .  's  is  easy.   In  the  above  example,  barring  x-|^2  ^"^"^  ^15 
assures  that  not  only  the  present  subtour,  but  also  the  reverse  subtour, 
is  eliminated  from  the  new  problem.   This  is  of  some  help  in  problems  of 
type  ELECT,  but  it  is  vital  in  problems  of  type  EUCL. 

Computational  Results  with  Eastman's  Method 

A  series  of  computer  runs  were  made  on  an  IBM  1620  Model  II  with  a 
con^Juter  program  embodying  Eastman's  method  with  elimination  of  redundant 
problems.   Source  language  was  FORTRAN  except  for  the  Assignment  Problem 
subroutine  and  the  redundant  problem  finder,  both  of  which  were  hand-coded 
in  (assembly  level)  SPS.   All  of  the  problems  were  of  type  RECT.   The 
Assignment  Problem  routine  had  neither  unwinding  or  reorganization.   The 
program  had  room  to  store  400  problems,  both  open  and  closed.   In  Table 
III  below,  the  symbol  OF  indicates  a  problem  which  overflowed  core  and 
could  not  be  solved. 
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Table  III 


Run  No. 


Time  in  Seconds  for  Travelling  Salesman  Problem 

of  size 


20x20 


40x40 


60x60 


80x80 


1 

2 

3 

4 

5 

6 

7 

8 

9 
10 
11 
12 
13 
14 
15 
16 
17 
18 

Runs 
Solved 
Mean  Time 
Median  Time 
Standard  Deviation 


110 
30 
10 
35 
15 

380 
35 
10 

115 
10 


10 
10 
75 
33 

108 


135 

110 

795 

165 

310 

5420 

270 

330 

950 

130 

100 

240 

165 

2655 

OF 

115 

625 

420 

250 

570 

210 

420 

OF 

1440 

260 

385 

480 

70 

225 

140 

705 

2220 
495 
OF 
160 
140 
700 
650 

6060 

10 

11 

18 

10 

10 

16 

198 

602 

1283 

165 

358 

573 

98 

712 

1769 

These  results  compare  favorably  with  Shapiro's  and  very  favorably  with 
Little's  for  problems  of  type  RECT.   But  there  are  possible  alternatives  to 
tlio  clioicc,  branching,  and  bounding  schemes  used  in  this  program.   Moreover, 
this  proj;ram  proved  nearly  worthless  for  problems  of  type  EUCL. 

We  now  proceed  to  investigate  some  alternatives  in  hope  of  improving 
performance. 
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Alternative  Bounding  Scheme  —  Little  Bounds 

In  step  E4,  we  set  the  lower  bound  of  each  new  problem  to  the  lower 
bound  (Assignment  Problem  solution  value)  of  its  parent.  It  is  possible 
to  develop  a  more  powerful  bound  by  a  scheme  due  to  Little.   If,  say,  x 


kn 


is  barred,  it  may  be  possible  to  find  new,  higher  values  of  u   and  v  : 

Thus,  after  c,   is  set  to  "^  : 
kn 

/  / 

u,   =  minCc.  .  -  v.)  v   =  min(c.   -  u.) 

k      .   kj    J  n      .    in    1 

J  1 


These  new  values  u,  and  v  maintain  dual  feasibility,  and  add  (u,  +  v 

k      n  •'  k    n 

u,  -  u  )  to  the  value  of  W.   This  new  value,  W  +  u,  4-  v   -  u,  -  v  ,  is  a 
k    n  k    u    k    u 

lower  bound  on  the  value  of  all  solutions  contained  in  the  new  problem. 
And,  of  course,  if  this  bound  equals  or  exceeds  "bogey",  the  new  problem 
need  not  be  stored  at  all. 

Alternative  Branching  Scheme  —  Link  Inclusions 

Eastman's  method  involves  the  elimination  of  sub tours  by  excluding 
links  (barring  x..'s).   It  is  possible  to  include  links  instead.   (Link 
inclusion  is  central  to  Little's  method;  Eastman  also  suggested  it). 
Here  two  new  problems  are  created  rom  the  parent.   One  of  these  new  prob- 
lems differs  from  its  parent  by  barring  an  x..;  the  other  is  created  by 
forcing  that  same  x..  to  be  non-zero  ("including"  a  link).   The  latter 
problem  must  also  bar  another  link — specifically,  that  x   which,  should 
it  appear  in  a  later  Assignment  Problem  solution,  would  form  a  subtour 
including  the  x   just  included. 
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When  branching  by  barring/inclusion,  new  problems  created  are  mutually 
exclusive.   Therefore,  there  are  no  redundant  problems  to  be  eliminated. 
Also,  when  solving  the  Assignment  Problem  or  taking  a  Little  bound,  columns 
and  rows  corresponding  to  included  links  must  be  ignored.   (This  corresponds 
to  Little's  scheme  of  "crossing  out"  rows  and  columns.) 

Choosing  a  Link  for  Inclusion 

When  branching  by  barring/inclusion,  there  is  a  question  as  to  which 
link  to  bar  and  include.   Probably  one  should  always  choose  an  x..  which 
is  non-zero  after  the  Assignment  Problem  Solution;  any  other  choice  will 
simply  lead  to  the  same  solution  appearing  again  in  the  new  problem  where 
X..  is  barred.   One  scheme,  requiring  very  little  computation,  is  to  fol- 
low the  chain  of  inclusions  starting  at,  say  City  1,  choosing  the  x   which 
would  extend  that  chain  one  more  city.   We  shall  call  this  scheme  "commit 
to  chain". 

Another  scheme  is  to  take  Little  bounds  corresponding  to  barring  each 
non-zero  x..  not  already  included.   We  then  choose  the  x..  producing  the 
largest  Little  bound.   We  shall  call  this  scheme  "commit  to  best". 

Alternative  Choice  Scheme  —  Move  to  the  Right 

In  Step  El,  we  choose  the  open  problem  with  lowest  lower  bound  (called 
"broad-front") .   There  are  computational  advantages  to  choosing  an  open 
problem  very  closely  related  to  the  one  just  solved,  since  the  Assignment 
Problem  procedure  can  be  entered  with  as  few  changes  as  possible  from  the 
previous  solution.   The  easy  way  to  do  this  is  to  choose  the  problem  most 
recently  opened.   We  call  this  scheme  "move  to  the  right". 
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Normally,  when  branching  by  link  inclusion,  one  chooses  the  new  prob- 
lem with  the  included  link,  since  it  is  "smaller"  in  the  sense  that  more 
rows  and  columns  can  be  ignored.   However,  in  either  type  of  branching,  one 
can  draw  a  Little  bound  for  each  new  problem,  then  choose  the  "best"  new 
problem  among  those  just  opened.   We  call  this  scheme  "move  to  best  of 
parent".   It  is  obviously  applicable  only  in  conjunction  with  "move  to  the 
right",  and  implies  taking  Little  Bounds. 

There  are  30  meaningful  combinations  of  all  these  alternatives,  sum- 
marized below: 
Choice  and  Bounding:   5  choices  marked  by  * 

0.  Choose  by  Broad-Front 

*0.0      Use  Parent  Bound  (BFPB) 

*0.1      Take  Little  Bound  (BFLB) 

1.  Move  to  the  Right 

*1.0      Use  Parent  Bound  (MRPB) 

*1.1      Take  Little  Bound  iMKLB) 

*1.2      Move  to  Best  of  Parent         (MRSB) 


Branching,  Barring,  and  Redundancy:   6  choices  marked  by  * 
0.       Branch  by  Barring  Only 
Bar  One  Link 

Store  All  New  Problems         (BlNR) 
Eliminate  Redundant  Problems    (BlER) 
Bar  Two  Links  if  Possible 
Store  All  New  Problems 


0.0 

*0.0.0 

*0.0.1 

0.1 

*0.1.G 
*0.1.1 


CB2NR) 


Eliminate  Redundant  Problems    (B2ER) 
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1.       Branch  by  Inclusion 
*1.0      Commit  to  Chain  (CMCH) 

*1.1      Commit  to  Best  (CMPR) 

Testing  Alternative  Schemes 

In  an  attempt  to  find  out  which,  if  any,  of  these  alternative  branch- 
ing, bounding,  and  choice  schemes  could  favorably  affect  performance,  six 
problems  were  chosen  to  be  run  all  30  ways.   Timings  for  these  runs,  made 
on  an  IBM  1130  at  the  Sloan  School,  are  displayed  in  Table  IV.   These  tim- 
ings are  not,  unfortunately,  comparable  to  those  shown  in  Tables  I  and  II, 
because  the  Assignment  Problem  subroutine  used  here  is  a  hand-coded  assem- 
bly-level version  which  is  approximately  three  or  four  times  as  fast  as 
the  program  timed  in  Tables  I  and  II.   Similarly,  these  timings  are  not 
comparable  to  those  of  Table  III  because  of  differences  between  machines. 

These  results  indicate  a  strong  effect  of  problem  type,  problem  size, 
and  problem  itself.   Beyond  this,  "commit  to  chain"  is  so  bad  as  to  be 
dismissed  out  of  hand. 

Taking  Little  Bounds  (BFLB,  MRLB)  almost  always  decreased  solution 
time  as  compared  to  using  the  parent  bound  tBFPB,  MRPB) .   "Move  to  best  of 
parent"  (MRSB)  was  responsible  for  a  dramatic  decrease  for  one  problem 
(No.  4),  but  otherwise  neither  helped  nor  hurt  very  much.   In  general, 
"Move  to  the  right"  was  faster  than  "broad-front",  but  for  one  problem 
(again  No.  4),  "broad-front"  was  superior  to  all  but  "move  to  best  of 
parent". 

Eliminating  redundant  problems  (BIER,  B2ER)  was  no  help,  except  in  the 
EUCL  problems,  probably  because  the  detection  of  redundant  problems  is  itself 
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Choice 

and  Bounding 

BFPB 

BFLB 

MRPB        MRLB 

MRSB 

1 

27 

23 

26         24 

36 

2 

8 

8 

7          7 

7 

BINR 

3 

4 

280 
16 

156 
19 

109         85 
95         86 

90 

19 

5 

205 

145 

50         43 

43 

6 
1 

148 

95 

46         38 

43 

30 

25 

25         24 

21 

2 

8 

8 

7          7 

7 

BIER 

3 

260 

156 

95         75 

70 

±f  1.1ml  L\ 

4 

18 

24 

45        43 

13 

5 

238 

177 

52         46 

45 

c 

6 

155 

119 

49         42 

40 

C 

1 
2 

28 
8 

24 
7 

26         24 
7          7 

37 

7 

crt 

B2NR 

3 

4 

280 
18 

160 
15 

110         90 
95         85 

90 
19 

,   Barring  ; 

5 

6 

131 
16 

85 
15 

40         34 
14         13 

36 

12 

1 
2 

31 
8 

26 
8 

25         25 
7         7 

39 

7 

00 

B2ER 

3 

275 

162 

100         75 

70 

•H 
J3 

4 

19 

26 

61         55 

18 

O 

c 

5 

170 

100 

42         37 

34 

CO 

6 

18 

12 

16         13 

13 

7 

280 

* 

120          80 

210 

2 

9 

8 

12          12 

12 

CMCH 

3 

4 

79 

* 
65 

*          * 
298        230 

* 
240 

5 

*^v 

220 

104         65 

70 

6 

**« 

** 

100         72 

66 

1 

39 

35 

43         34 

34 

2 

13 

9 

12         10 

10 

CMPR 

3 

130 

110 

90         70 

68 

4 

50 

32 

82         60 

61 

5 

61 

40 

37         28 

28 

6 

47 

27 

22         18 

21 

*  Longer  than  300  seconds 
**  Blew  up  —  i.e.  exceeded  core 


Table  IV 


Problems  1-2  20-city  RECT 
3-4  40-city  RECT 
5-6   10-city  EUCL 
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time-consuming.   "Bar  two  if  possible"  (B2NR,  B2ER)  was  of  great  help  in 

the  EUCL  problems,  because  of  the  elimination  of  reverse  subtours.   But 

the  clear  cliampion  branching  scheme  was  "commit  to  best",  (CMPR) .   It  should 

be  noted  that  this  scheme  differs  from  Little's  only  in  that  it  chooses  from 

among  those  x..'s  set  non-zero  by  the  Assignment  Problem  procedure,  rather 

than  from  among  all  ..'s  where  c..  =  u.  +  v.. 

ij  ij    1    J 

In  summary,  the  best  procedure  (for  these  six  problems)  was  a  toss-up 
between  CMPR/MRLB  and  CMPR/MRSB,  with  B2ER  a  good  alternative  to  CMPR  for 
the  EUCL  problems. 

More  Extensive  Tests  of  CMPR/MRSB  for  RECT  Problems 

Forty  RECT  problems  (ten  each  20,  40,  60  and  80  cities)  were  timed 
using  "commit  to  best"  and  "move  to  best  of  parent".   These  timings  are 
displayed  in  Table  V.   Runs  were  made  under  conditions  identical  with 
those  for  Table  IV.   (Timings  for  60  and  80-city  problems  are  to  the  near- 
est 5  seconds.) 

Table  V 
T.ime  in  Seconds  for  Travelling  Salesman  Problems  (RECT)  of  size 

Prob  20x20        40x40        60x60        80x80 


1 

34 

91 

230 

110 

2 

11 

30 

185 

890 

3 

6 

33 

115 

225 

4 

13 

22 

710 

455 

5 

8 

92 

90 

230 

6 

17 

36 

130 

995 

7 

13 

90 

160 

260 

8 

8 

65 

210 

145 

9 

23 

118 

295 

435 

10 

11 

29 

45 

210 

Runs 

10 

10 

10 

10 

Solved 

10 

10 

10 

10 

Mean  Time 

14 

61 

215 

395 

Median  Time 

12 

51 

180 

245 

Standard  Deviation 

8 

33 

175 

295 
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IV.   SOLUTION  OF  THE  TRAVELLING  GOURMET  PROBLEM 
BY  EXTENSION  OF  EASTMAN'S  METHOD 

All  of  the  discussion  in  the  previous  section  can  be  straightforward 
extended  to  the  Travelling  Gourmet  Problem.  All  that  is  required  is  the 
ability  to  recognize  the  violation  of  a  cluster  constraint  and  to  define 
new  subproblems  in  which  the  same  violation  cannot  occur.   We  then  must 
modify  two  steps  of  the  general  procedure: 

EIG.   Choice:   choose  an  open  problem  (by  BFPB ,  BFLB,  MRPB ,  MRLB, 
or  MRSB) .   Mark  it  closed.   If  the  included  links  (applicable  to 
CMPR  only)  violate  any  cluster  constraint,  go  directly  to  Step  E7 ; 
otherwise  go  on  to  Step  E2. 

E3G.   Branching:   If  the  Assignment  Problem  Solution  contains  sub- 
tours,  split  the  problem  (by  BlNR,  RIER,  E2NR,  or  CMPR)  and  go  on 
to  Step  E7. 

If  the  Assignment  Problem  solution  has  no  subtours  but  does  violate 
any  cluster  constraint,  treat  the  chain  of  links  violating  the  constraint 
almost  exactly  as  a  subtour.   The  last  S,  links  in  the  chain  tie  together 
S   +  1  cities,  thus  violating  the  cluster  constraint.   Split  the  problem 
Into  S   new  problems,  each  of  which  has  one  of  these  S,  offending  links 
barred.   Do  not  bar  any  included  link.   Then  go  to  step  E7 . 

It'  neither  the  tour  constraint  nor  any  cluster  constraint  is  violated, 
go  to  Step  E6  to  record  a  new  bogey. 
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The  first  problem  chosen  to  be   solved  by   this   technique  was   an  11  - 
"city"   problem  made   up   of   distances   between   11   3-star   restaurants    in 
France.      "Cities"   5,6,7,   and   10  are   located   in  Paris.      "Cities"   2,8,9, 
and   11  are   restaurants  boasting  predominantly  Lyonnaise  cuisine. 

The   cost   matrix   is   shown   in  Table   VI.      Two   clusters  were  defined: 
K     =   (5,6,7,10),    S      =   2;    and  K     =    (2,8,9,11)    S      =   2.      The  problem  was 
first   solved   as   a  Travelling   Salesman  Problem  using  CMPR/MRSB    ("commit 
to  best",    "move    to  best   of   parent").      Then   the   simple  additions  were 
made   to  activate   the  cluster  constraints  as   shown  above.      The   increase 
in  processing   time  was  very  large,   but  a   solution  was   obtained.      Table  VII 
summarizes    these    two   runs.      Conditions   were   the   same   as   for   Tables    IV  and  V. 

Table   VI 

COST    KAT'-'  1  X 

(;<;t)9  1  2  3  A  S  6  7  b  9  10  11 

1  9q^-^       ,,3o       uM       bfi       ^^e       ^'*<i       ^^6       ^05       ^33       'VAP       S92 

2  43h  <i  J'V)  '30A  ^^Z  SAb  5A6  5A6  174  302  5A6  ?.bl 

3  ^33  "bSA  ^^>99  701  23A  ?3A  ?3^  433  Aol  234  346 
A  ^M  4h2  i'Ul  9999  (25  725  72")  26b  240  72*?  3S5 
<3  ^i,p  <,.^(,  ,  iA  72  5  9999  0  0  4-j9  490  0  3b9 
(,  ^^;,  ',^M  234  725  0  9')99  0  459  490  0  3C9 
7  i,t,y  <>^(.  /  >,^  725  0  »•  9'y  r/  A59  490  0  3b9 
H  ^,o''  !/.  4)3  ?l<^  4'^.9  ^5  y  '-,59  9999  28  459  b7 
9  '.  )  ^  302  '.61  240  490  490  490  28  9999  490  115 

10  4't8        5h6        /i4        725  0  0  0       459        490     9999        339 

11  S92       /t  I        i46       355       389       3rt9       3m9  b/        11^       369    9999 


^  38  T 


Table  VII 


11  -  "City"  Problem  Solved  as; 


Time  in  Seconds  (approx) 

Solution  Value 

Subproblems  Evoked 

Discarded  by  Little  Bounds 

Discarded  by  Cluster  Constraints 

Discarded  by  New  Bogeys 

Submitted  to  Assignment  Problem  Solver 

Discarded  by  Assignment  Problem  Solver 

Completed  by  Assignment  Problem  Solver 

Optimal  Found  in  Step 


Travell 

ing 

Travelling 

Salesman 

Gourmet 

40 

550 

2261 

km 

2538  km 

1101 

13190 

536 

5367 



640 

3 

5 

562 

7178 

10 

352 

552 

6826 

128 

981 
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V.   COMMENTS  ON  RESULTS 

First,  Eastman's  method,  as  modified  and  extended,  is  clearly  very 
powerful  for  the  Travelling  Salesman  Problem  with  uniform  random  costs 
(type  RECT) .   We  know  of  no  other  exact  solution  of  problems  as  large  as 
80  X  80,  nor  of  times  so  low.   Perhaps  more  important.  Little  found  that 
his  technique  suffered  an  increase  in  time  of  about  lOX  for  every  10 
extra  cities.   The  results  in  Tables  III  and  V  suggest  an  increase  of  at 
most  2X  for  every  10  extra  cities  using  Eastman's  scheme. 

Second,  we  find  (like  Eastman,  Little  and  Shapiro)  that  symmetric, 
and  especially  Euclidean  problems  are  very  difficult:   a  10-city  Euclidean 
problem  is  about  as  hard  to  solve  as  a  40-city  RECT  problem.   This  is 
because  of  the  large  number  of  subtours  of  length  2  found  in  Assignment 
Problem  solutions  under  symmetric  cost  matrices.   Eastman  suggested  an 
extension  to  his  method  for  the  symmetric  case.   His  scheme,  among  others, 
is  the  subject  of  the  next  paper. 

Third,  Branch  and  Bound  techniques  and  Network-Flow  techniques  are 
both  recognized  and  in  constant  use.   Put  together,  they  can  be  highly 
S3mergistic.   This  class  of  algorithms  is  a  case  in  point. 
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NOTES 


1.  See  Ford,  L.R.  Jr.  and  D.R.  Folkerson,  Flows  in  Networks,  Princeton, 
N.J.:   Princeton  University  Press  (1962). 

2.  See  Dantzig,  G.B.,  Linear  Programming  and  Extensions,  Princeton, 
N.J.:   Princeton  University  Press  (1963),  Chapter  15. 

3.  See  Bellmore,  M.  and  G.L.  Nemhauser,  "The  Travelling  Salesman 
Problem:   A  Survey",  Operators  Research  16,  538-558  (1968). 

4.  I  am  indebted  to  D.N.  Ness  for  naming  this  problem. 

5.  See  Dantzig 

6.  See  Dantzig 

7.  See  Pierce,  J.F.  and  D.  Hatfield,  "Production  Sequencing  by  Com- 
binational Programming"  in  Operators  Research  and  the  Design  of 
Management  Information  Systems,  New  York,  TAPPl  (1967). 

8.  "3-Star"  as  defined  by  Guide  Michelin. 

10.  See  Hadley,  G. ,  Linear  Programming,  Reading,  Massachusetts: 
Addison-Wesley  C1962)  Section  10-9. 

11.  Little  coined  the  term. 

12.  Little,  J.D.C.,  K.G.  Murty,  D.W.  Sweeney,  and  C.  Karle,  "An 
Algorithm  for  the  Travelling  Salesman  Problem",  Operation  Research 
11,  969-989  (1963). 

13.  Eastman,  W.L.  "Linear  Programming  with  Pattern  Constraints",  Ph.D. 
Dissertation,  Harvard,  1958. 

14.  Shapiro,  D. ,  "Algorithms  for  the  Optimal  Cost  Travelling  Salesman 
Problem",  Sc.D.  Thesis,  Washington  University,  St.  Louis  (1966). 
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